Information processing apparatus, information processing method, and storage medium

ABSTRACT

An information processing apparatus acquires first physical makeup information representing a physical makeup of a first object. The first physical makeup information is information generated based on images obtained by imaging the first object with a plurality of imaging apparatuses in a plurality of directions. The information processing apparatus acquires second physical makeup information representing a physical makeup of a second object and motion information representing motion of a plurality of parts of the second object. The information processing apparatus generates motion information associated with the first physical makeup information based on the second physical makeup information and the motion information relating to the second object.

BACKGROUND Field

The present disclosure relates to a technique to process informationrelating to a physical makeup and motion of an object.

Description of the Related Art

Some sports participants (“learners”) correct their form based on a formof a model proficient in the particular sport in question. For example,a golfer can work on correcting their form by comparing an image,captured by a camera, of their form with an image of a form of a model.

Japanese Patent No. 4646209 discusses superimposing a skeleton imagerepresenting a model form drawn with a wire frame, which is internallytranslucent or transparent, on a moving image of a subject anddisplaying the result to facilitate comparison of the form of thesubject with the model form. Japanese Patent No. 4646209 discussesadjusting a position, a direction, a size, etc. of the skeleton imagevia a user operation, which enables to more correctly superimpose theskeleton image on the moving image of the subject.

The technique discussed in Japanese Patent No. 4646209, does not in somecases enable the subject to realize a posture to achieve the mostproficient motion represented by the model form. For example, in a casewhere a physical makeup of the subject and a physical makeup of themodel are different from each other, the learner cannot take a sameposture as the model form, and cannot achieve the proficient motion evenwhen the learner forcibly takes a posture close to the model form.

SUMMARY

The present disclosure is directed to providing information useful for aperson (e.g., learner) to achieve motion of another person (e.g.,model).

According to an aspect of the present disclosure, an informationprocessing apparatus includes a first acquisition unit configured toacquire first physical makeup information representing a physical makeupof a person as a first object, the first physical makeup informationbeing generated based on images obtained by imaging the first objectwith a plurality of imaging apparatuses in a plurality of directions, asecond acquisition unit configured to acquire second physical makeupinformation representing a physical makeup of a person as a secondobject and motion information representing motion of a plurality ofparts of the second object, and a generation unit configured to generatemotion information associated with the acquired first physical makeupinformation based on the acquired second physical makeup information andthe acquired motion information representing the motion of the pluralityof parts of the second object.

Further features will become apparent from the following description ofexemplary embodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of a configuration of ananalysis system according to an exemplary embodiment.

FIG. 2 is a diagram illustrating imaging with a camera group accordingto the exemplary embodiment.

FIG. 3 is a diagram illustrating an example of a data estimationgraphical user interface (GUI) according to the exemplary embodiment.

FIG. 4 is a diagram illustrating an example of a form correction GUIaccording to the exemplary embodiment.

FIG. 5 is a diagram illustrating an example of a data display settingGUI according to the exemplary embodiment.

FIG. 6 is a flowchart illustrating operation of an informationprocessing apparatus according to the exemplary embodiment.

FIG. 7 is a flowchart illustrating operation data estimation processingto be performed by the information processing apparatus according to theexemplary embodiment.

FIG. 8 is a flowchart illustrating estimated data correction processingto be performed by the information processing apparatus according to theexemplary embodiment.

FIG. 9 is a flowchart illustrating data display setting processingaccording to the exemplary embodiment.

FIG. 10 is a block diagram illustrating an example of a functionalconfiguration of the information processing apparatus according to theexemplary embodiment.

FIG. 11 is a diagram illustrating rotation transfer according to theexemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

An exemplary embodiment of the present disclosure is described withreference to drawings. The following exemplary embodiment is not seen tobe limiting, and all of combinations of features described in thepresent exemplary embodiment are not necessarily essential for solutionsprovided by the present disclosure. Like components are described withlike numerals.

[Configuration of Analysis System]

FIG. 1 is a diagram illustrating an example of a configuration of ananalysis system 10 according to the present exemplary embodiment. Theanalysis system 10 illustrated in FIG. 1 includes an informationprocessing apparatus 100 and a plurality of imaging apparatuses (cameragroup) 109. The information processing apparatus 100 includes a centralprocessing unit (CPU) 101, a main memory 102, a storage unit 103, aninput unit 104, a display unit 105, and an external interface (I/F) unit106, where these units are connected to each other via a bus 107.

The CPU 101 is an arithmetic processing unit that controls theinformation processing apparatus 100, and executes various types ofprograms stored in the storage unit 103, etc. to perform variousprocessing. The information processing apparatus 100 can include one ora plurality of types of dedicated hardware different from the CPU 101,and the dedicated hardware can execute at least a part of the processingto be executed by the CPU 101. Examples of the dedicated hardwareinclude an application specific integrated circuit (ASIC), a fieldprogrammable gate array (FPGA), and a digital signal processor (DSP).The main memory 102 temporarily stores data, parameters, etc. used invarious kinds of processing, and provides a work area to the CPU 101.The storage unit 103 is a large-capacity storage unit that storesvarious types of data necessary for various types of programs and fordisplay of graphical user interface (GUI). For example, a hard disk or anonvolatile memory such as a silicon disc is used.

The input unit 104 includes a device such as a keyboard, a mouse, anelectronic pen, and a touch panel, and receives operation input from auser. The display unit 105 includes a liquid crystal panel, and displaysanalysis result on a GUI, etc. The external I/F unit 106 is connected toexternal apparatuses such as cameras included in the camera group 109via a local area network (LAN) 108, and performs transmission/receptionof image data and control signal data. The bus 107 connects theabove-described units, and performs data transfer. The camera group 109is connected to the information processing apparatus 100 via the LAN108, and starts or stops imaging, changes camera settings (shutterspeed, diaphragm, etc.), and transfers captured image data, based oncontrol signals from the information processing apparatus 100. Thecamera group 109 includes one or more cameras, and the number of camerasis not limited.

The configuration of the analysis system 10 is not limited to theabove-described configuration. For example, the information processingapparatus 100 may not be connected to the camera group 109. Theinformation processing apparatus 100 can be connected to a storagedevice (not illustrated) that stores data based on the imaging by thecamera group 109 via the LAN 108, and can acquire data based on theimaging from the storage device. For example, the information processingapparatus 100 may not include one or more of the input unit 104 or thedisplay unit 105. At least the input unit 104 or the display unit 105can be provided as a separate unit external to the informationprocessing apparatus 100, and the CPU 101 can operate as an inputcontrol unit controlling the input unit 104 and as a display controlunit controlling the display unit 105.

FIG. 10 is a diagram illustrating an example of a functionalconfiguration of the information processing apparatus 100 according tothe present exemplary embodiment. The information processing apparatus100 includes an imaging determination unit 1001, a data superimpositionunit 1005, a data comparison unit 1009, a data editing unit 1010, adisplay method setting unit 1011, a user input unit 1012, an estimationunit 1013, and a correction unit 1014. The estimation unit 1013 includesan operation estimation unit 1003 and a skinning unit 1004. Thecorrection unit 1014 includes a position correction unit 1006, a timesynchronization unit 1007, and a deficiency interpolation unit 1008.These functional units included in the information processing apparatus100 are implemented when the CPU 101 executes various types of programsstored in the storage unit 103, etc. Any of these functional units canbe implemented by the dedicated hardware. The detail of each of thefunctional units included in the information processing apparatus 100 isdescribed below with reference to a flowchart.

FIG. 2 is a diagram illustrating an installation example of the camerasincluded in the camera group 109. While there are many objects that canbe analyzed, for example, sports and acting performances, an example ofanalyzing a golf swing will be described in the present exemplaryembodiment. In the following description, the term “learner” refers toan individual who is trying to achieve the motion of another person, whomay be referred to as a “model”. In FIG. 2, a learner as an object 202is located in a space 201, and six cameras 203 are installed to imagethe object 202 in a plurality of directions. Imaging parameters such asa camera direction, a focal length, and exposure control parameters areset in each of the cameras 203 included in the camera group 109 suchthat the whole of the space 201 or a region of interest including theobject 202 in the space 201 is included in a viewing angle of each ofthe cameras 203.

[Description of GUI]

FIG. 3, FIG. 4, and FIG. 5 are diagrams each illustrating an example ofa GUI displayed by the analysis system 10 according to the presentexemplary embodiment. These screens are displayed on the display unit105 of the information processing apparatus 100. Alternatively, thesescreens can be displayed on another display apparatus (not illustrated)connected to the information processing apparatus 100. FIG. 3illustrates a basic screen of the GUI that is an example of a screendisplayed in data estimation. A data estimation GUI 300 includes animage display area 301, an analyzed data display area 302, an imagingbutton 303, a data estimation button 304, and a data superimpositionbutton 305. The data estimation GUI 300 also includes a positioningbutton 306, a time synchronization button 307, a deficiencyinterpolation button 308, a form correction shifting button 309, adisplay setting shifting button 310, and a seek bar 311.

FIG. 4 illustrates an example of a screen that is displayed to correctthe form by the learner. A form correction GUI 400 includes an imagedisplay area 401, a different viewpoint display area 402 for formcorrection, a seek bar 403, a data estimation shifting button 404, and adisplay setting shifting button 405. FIG. 5 illustrates an example of ascreen that is displayed for display setting of data. A data displaysetting GUI 500 includes an image display area 501, an edited datadisplay area 502, a display setting area 503 for model data, a displaysetting area 504 for learner data, and a display setting area 505 foredited data. The data display setting GUI 500 also includes a seek bar506, a data estimation shifting button 507, and a form correctionshifting button 508.

Operation on the GUI and operation of the information processingapparatus 100 in response to the operation on the GUI are describedbelow. The operation on the GUI is performed by the user with, forexample, the mouse and the touch panel. When the imaging button 303 ispressed in the data estimation GUI 300, the information processingapparatus 100 outputs an instruction, to the camera group 109, to startcapturing moving images of the learner's swing from multiple viewpoints.When the imaging button 303 is pressed again while the camera group 109is capturing the moving images, the information processing apparatus 100outputs an instruction to end capturing the moving images to the cameragroup 109.

When the data estimation button 304 is pressed after the camera group109 ends capturing the moving images, the information processingapparatus 100 estimates three-dimensional shape data (hereinafter,referred to as shape data) and skeleton data of the learner based on thecaptured images, and the image of the learner based on the estimatedresult is displayed in the image display area 301. In the presentexemplary embodiment, the shape data about the object such as thelearner and a model represents an outer shape of the object, and theskeleton data about the object represents a bone structure of theobject. The skeleton data representing the bone structure of the objectis expressed by, for example, markers indicating positions of aplurality of joints of the object and line segments connecting themarkers. For example, the image displayed in the image display area 301is an image captured from the viewpoint of any of the cameras includedin the camera group 109. In the present exemplary embodiment, the imagecaptured from the viewpoint of the camera (tentatively, Cam 7) thatcaptures images of the learner from a front side is displayed.

When the data superimposition button 305 is pressed, the informationprocessing apparatus 100 displays an image representing motion of themodel by superimposing the image representing motion of the model on theimage of the learner displayed in the image display area 301. At thistime, the information processing apparatus 100 can display an imagerepresenting shape data and skeleton data about the model, or display animage representing a result obtained by applying motion information onthe model to the skeleton data about the learner. Applying the motioninformation on the model to the skeleton data about the learner makesenables generating an image representing a state where the motion of themodel is reproduced with the physical makeup of the learner. When theinformation processing apparatus 100 displays the images beingsuperimposed, the information processing apparatus 100 can transparentlyor translucently display one or more of the image of the learner or theimage representing the motion of the model.

The information processing apparatus 100 displays information relatingto the motion of the learner and the information relating to the motionof the model in the analyzed data display area 302 so that theinformation relating to the motion of the learner can be compared withthe information relating to the motion of the model. The contents to bedisplayed here include change of an angle of a specific joint in eachobject with time, and an inflection point in the change of the angle.The angle information on the joint to be displayed can be selected basedon the user operation, or the information processing apparatus 100 canautomatically select a joint of which the motion is largely differentbetween the learner and the model.

When any of the positioning button 306, the time synchronization button307, and the deficiency interpolation button 308 is pressed, theinformation processing apparatus 100 corrects one or more of the modeldata or the learner data in order to enable the learner to easilycompare the form of the learner with that of the model. At this time, ifan automatic check box displayed next to each of the buttons is checked,the corresponding correction is automatically performed. If theautomatic check box is not checked, the correction is performed inresponse to user operation to perform manual correction. The correctionresult of the data is reflected on the image in the image display area301 and the image in the analyzed data display area 302.

When operation to move the seek bar 311 is performed, the informationprocessing apparatus 100 displays an image corresponding to a time pointbased on a position of the seek bar 311 in the image display area 301. Aposition corresponding to a specific time point when, for example,motion difference between the learner and the model is large can bedisplayed on the seek bar 311. For example, in FIG. 3, a circle on theseek bar 311 indicates a time when the motion difference between thelearner and the model is large, specified by the information processingapparatus 100. When the form correction shifting button 309 is pressed,the information processing apparatus 100 shifts the displayed screen tothe form correction GUI 400. When the display setting shifting button310 is pressed, the information processing apparatus 100 shifts thedisplayed screen to the data display setting GUI 500.

When the screen is shifted to the form correction GUI 400, an imagesimilar to the image in the image display area 301 is displayed in theimage display area 401. An image captured from the viewpoint at whichthe motion difference between the learner and the model is the largestis displayed in the different viewpoint display area 402. The learnercan correct the form while viewing the data displayed in the area. Thefunctions of the seek bar 403 and the display setting shifting button405 are similar to those described in the description regarding the dataestimation GUI 300. When the data estimation shifting button 404 ispressed, the information processing apparatus 100 shifts the displayedscreen to the data estimation GUI 300.

When the screen is shifted to the data display setting GUI 500, theshape data and the skeleton data about the model and the learner aredisplayed in the image display area 501. The information processingapparatus 100 edits the shape data and the skeleton data about thelearner based on the user operation in the data display setting GUI 500,and displays the edited shape data and the edited skeleton data in theimage display area 501. Only the shape data or the skeleton data can bedisplayed, or only the shape data or the skeleton data can be editable.When a joint of the object (model or learner) in the image display area501 is designated by the user operation, a bent state of the joint, etc.is quantitatively displayed in the image display area 501. A graph ofthe bent state of the joint designated in the image display area 501 isdisplayed in the edited data display area 502. The user can display astate where the learner performs motion similar to the motion of themodel in the image display area 501 by editing the shape data and theskeleton data about the learner to bring the state of the joint of thelearner and the state of the joint of the model closer to each otherwhile viewing the edited data display area 502. The informationprocessing apparatus 100 can automatically edit the shape data and theskeleton data about the learner to bring the state of the jointdesignated by the user operation closer to the state of the joint of themodel, and can also automatically edit a joint not designated.

An image is displayed in the display setting area 503 for model data,the display setting area 504 for estimated data, and the display settingarea 505 for edited data so that the user selects displaying or notdisplaying various kinds of data and then selects a display format in acase where the user selects displaying the various kinds of data. Forexample, when a shape of the object is selected as a display target,texture is selected as the display format, and transparency is set to anintermediate value (50%) as illustrated in the display setting area 504for estimated data in FIG. 5, the shape data is displayed in translucentcolor (with texture). The functions of the seek bar 506, the dataestimation shifting button 507, and the form correction shifting button508 are similar to those described in the description about the dataestimation GUI 300 and the form correction GUI 400.

[Flow of Processing by Information Processing Apparatus]

The processing to be performed by the information processing apparatus100 is described with reference to flowcharts illustrated in FIG. 6,FIG. 7, FIG. 8, and FIG. 9 and a block diagram illustrated in FIG. 10.The processing illustrated in these flowcharts is achieved when the CPU101 reads predetermined programs from the storage unit 103 and loads theprograms to the main memory 102, and the CPU 101 executes the loadedprograms. FIG. 10 illustrates the functional configuration of theinformation processing apparatus 100, and the functional units areimplemented when the CPU 101 executes the above-described programs. Atleast a part of the processing in the flowcharts and the functionalunits of the information processing apparatus 100 can be implemented byone or a plurality of types of dedicated hardware different from the CPU101.

The outline of the processing to be performed by the informationprocessing apparatus 100 is described with reference to the flowchartillustrated in FIG. 6 and the block diagram illustrated in FIG. 10. Theprocessing illustrated in FIG. 6 can be started at timing when theinitial setting of the analysis system 10 including the camera group 109is completed. The start timing of the processing illustrated in FIG. 6,however, is not limited to the above-described timing. For example,after the data obtained by imaging the object such as the learner in theplurality of directions is input to the information processing apparatus100, the processing illustrated in FIG. 6 can be performed at a timingwhen operation to display the image for form comparison is performed bythe user.

In step S601, the operation estimation unit 1003 generates operationdata (data about shape and skeleton motion) about the object based onthe moving image input to the image input unit 1002, and the skinningunit 1004 associates the shape data and the skeleton data with eachother. The detail of the processing in step S601 is described below withreference to FIG. 7. In step S602, the data superimposition unit 1005reads and acquires operation data about the model that is to be comparedwith the operation data about the learner. The operation data about themodel includes information representing the physical makeup (shape dataand skeleton data) of the model and information representing motion ofthe model. In the present exemplary embodiment, the informationrepresenting the motion of the object such as the model and the learneris represented as information representing motion of the skeletonincluding a plurality of joints and connection parts thereof, namely, asmotion information representing the motion of the plurality of joints ofthe object. The contents of the motion information are not limitedthereto, and the information may represent motion of a part differentfrom the joint, for example, a middle part between the joints. In thepresent exemplary embodiment, the case where the information processingapparatus 100 acquires and processes both the shape data and theskeleton data about the object as the information representing thephysical makeup of the object is described. The processing, however, isnot limited thereto, and the information processing apparatus 100 canacquire at least one of the shape data or the skeleton data, and performsimilar processing.

The model data read by the data superimposition unit 1005 can beautomatically selected with reference to attribute information (age,gender, body height, weight, etc.) input by the user. The target datacan be selected by the user from a plurality of pieces of data stored ina database. The data to be read can be selected using an attribute ofthe learner determined based on the captured image. The model data canbe previously generated based on images obtained by imaging the model bya plurality of cameras in a plurality of directions, or can be generatedby other computer graphics processing.

In step S603, the position correction unit 1006 and the timesynchronization unit 1007 perform positioning and time synchronization,respectively, on the operation data about the learner and the operationdata about the model. The deficiency interpolation unit 1008 performsdeficiency interpolation if there is a deficiency in the data. Thedetail of the processing in step S603 is described below with referenceto FIG. 8. These processing can be performed selectively, and executionof all of the processing is not essential. In the time synchronization,the model data can be adjusted to match with the learner data as opposedto processing to adjust the learner data to match with the model data.In the present exemplary embodiment, a case where the learner data isadjusted to match with the model data is described.

In step S604, the data comparison unit 1009 performs comparison of thedata, and displays a result of the comparison in a display area of theGUI. At this time, the data editing unit 1010 and the display methodsetting unit 1011 can perform editing of the data and setting of thedisplay method, respectively, and the results are reflected on the imagedisplayed in the display area. The detail of the processing in step S604is described below with reference to FIG. 9. In step S605, the imagingdetermination unit 1001 determines whether to perform imaging again bythe camera group 109 based on an input to the user input unit 1012. In acase where the imaging is not performed again (NO in step S605), theprocessing ends. In a case where the imaging is performed again (YES instep S605), the imaging determination unit 1001 transmits an imaginginstruction to the camera group 109 via the LAN 108, and the imaging isthen started.

The details of the above-discussed steps are described with reference tothe flowcharts illustrated in FIG. 7, FIG. 8, and FIG. 9, and the blockdiagram illustrated in FIG. 10. At this time, the processing by each ofthe functional units is executed when the user input unit 1012 receivesa user input to the above-described GUI and an instruction correspondingto the input is output to each of the units.

The generation of the operation data in step S601 is described withreference to FIG. 7 and FIG. 10. FIG. 7 illustrates detailed flow of theprocessing in step S601. In step S701, the imaging determination unit1001 transmits an imaging instruction to the camera group 109 via theLAN 108 in response to pressing of the imaging button 303. The pluralityof cameras included in the camera group 109 images the learner in theplurality of directions in response to the instruction. Moving imagesacquired by imaging the learner are read to the main memory 102 via theLAN 108, the external I/F unit 106, and the bus 107. When the imagingbutton 303 is pressed again, imaging of the moving images is stopped.

In step S702, the operation estimation unit 1003 reads the moving imagesbased on the above-described imaging from the main memory 102 inresponse to pressing of the data estimation button 304, and generatesthe operation data about the learner based on the moving images. Theoperation data generated at this time includes information representingthe physical makeup (estimated result of shape data and skeleton data)of the learner and information representing motion (estimated result ofskeleton motion) of the learner. The generated operation data is outputto the skinning unit 1004. Examples of the method of estimating theshape of the object from the images captured in the plurality ofdirections include a visual cone intersection method. The visual coneintersection method is a method in which silhouettes of an object areextracted from images captured from a plurality of viewpoints, and thesilhouettes are projected to a three-dimensional space to determineintersections, thereby acquiring three-dimensional shape data about theobject. The method of estimating the shape of the object is not limitedthereto, and various methods can be used.

In the present exemplary embodiment, as the method of estimating theskeleton data from the moving images, a method of estimating theskeleton data with use of the above-described estimated shape data isdescribed. Rough irregularities of a human shape are fixed depending onparts of the body. For example, a human structure in which a neck isthinner than a head and a part lower than the neck is branched intoright and left shoulders and a chest, is typical. Information on suchrough irregularities can be used to estimate the skeleton data. When across-sectional shape is scanned from the head to feet of a standinghuman, a structure in which the cross-sectional shape is graduallyexpanded/contracted and branched can be confirmed. From such astructure, joints of the head, the neck, the shoulders, elbows, wrists,a vicinity of a pelvis, thighs, knees, and ankles are calculated, andthe skeleton data is generated so as to connect the joints. The methodof estimating the skeleton data, however, is not limited to theabove-described method.

In step S703, the skinning unit 1004 associates the shape data and theskeleton data with each other based on the operation data output fromthe operation estimation unit 1003, and outputs a result of theassociation to the data superimposition unit 1005. Skinning that isprocessing performed by the skinning unit 1004 is processing toassociate the skeleton motion and deformation of vertices of a surfaceshape of the object corresponding to the skeleton with each other. Asone of the skinning methods, there is a method of determining adeformation degree of the shape based on a distance from a joint locatedat a root of the skeleton to the vertex of the surface shape. Forexample, the vertex at a position three-dimensionally close to the jointis slightly moved, and the vertex at a position three-dimensionally farfrom the joint is largely moved. In a case where the movement of thevertex is determined based only on the three-dimensional distance,however, a defect such as movement of the vertex of an abdomen inresponse to movement of the joint of an arm can occur. For this reason,a method of appropriately performing skinning using a geodesic distanceand the three-dimensional distance on the shape data is also used. Theseare examples of the skinning processing, and the specific contents ofthe processing by the skinning unit 1004 are not limited thereto.

In step S704, the imaging determination unit 1001 determines whether toperform imaging by the camera group 109 again. For example, in a casewhere a failure degree of any of the shape data and the skeleton dataestimated in step S702 is greater than a predetermined threshold, theimaging determination unit 1001 can transmit the imaging instruction tothe camera group 109, and the imaging can be repeated. For example, theinformation processing apparatus 100 can display a screen for selectingeither performing or not performing the imaging again, and can transmitan instruction to perform the imaging again in response to the useroperation. In a case where the imaging is to be performed again (YES instep S704), the processing is returned to step S701, and imaging of themoving images and estimation of the shape data and the skeleton data areperformed again. In the present exemplary embodiment, the imagingdetermination unit 1001 determines in step S704 whether to perform theimaging again as an example, but the information processing apparatus100 can detect pressing of the imaging button 303 at optional timing andoutputs the instruction for the imaging again.

After the processing in step S601 ends, the model data is read in stepS602, and the learner data and the model data are displayed. To displaythe data, a plurality of methods is considered. For example, there is amethod in which an unprocessed image representing the shape and theskeleton motion of the model is superimposed on the image of thelearner. For example, there is a method of displaying the image of thelearner with the motion of the model by applying the skeleton motion ofthe model to the skeleton data of the learner. Various additionaldisplay methods can also be considered. In the present embodiment, acase where the image representing the motion of the learner and theimage representing the shape of the learner with the motion of the modelare superimposed and displayed is described. As described above, theshape data and the skeleton data are associated with each other in stepS703. Accordingly, when the data representing the skeleton motion of themodel is transferred onto the skeleton data of the learner, the shapedata about the learner is deformed based on the motion, and the imagerepresenting the shape of the learner with the motion of the model canbe acquired.

The image representing the motion of the learner, which is one of theimages to be displayed, may be a captured image obtained by imaging thelearner by a camera, or an image (e.g., image representing skeletonmotion) based on the motion information on the learner generated byprocessing in step S601. In the present exemplary embodiment, a casewhere, as the image corresponding to the motion of the model that is theother of the images to be displayed, an image representing thethree-dimensional shape of the learner performing the motion of themodel is displayed is described. The image, however, is not limitedthereto, and the skeleton image of the learner performing the motion ofthe model may be displayed. The information processing apparatus 100 candisplay, on the display unit, the image corresponding to a display modeselected from a plurality of display modes for displaying theabove-described various contents. In this way, an image easily visibleby the user can be provided.

The transfer of the skeleton motion is described with reference toFIG. 1. An initial joint position of the model is denoted by H_(EP), ajoint position of the model after operation is denoted by H_(E), aninitial joint position of the learner is denoted by H_(LP), and a jointposition of the learner after operation is denoted by H_(L). A rotationmatrix R_(θ) that represents rotation of a joint is defined. Therotation matrix with respect to the joint of the model is defined by thefollowing expressions (1) to (3).

{right arrow over (H _(E1) H _(E2))}=R _(θ1){right arrow over (H _(Ep1)H _(Ep2))}  (1)

{right arrow over (H _(E2) H _(E3))}=R _(θ2){right arrow over (H _(E1) H_(E2))}  (2)

{right arrow over (H _(E2) H _(E3))}=R _(θ2) R _(θ1){right arrow over (H_(Ep1) H _(Ep2))}  (3)

It is assumed that a vector representing the skeleton between the jointsis normalized. When this is transferred to the joint of the learner, therotation of the joint is represented by expressions (4) and (5).

{right arrow over (H _(L1) H _(L2))}=R _(θ1){right arrow over (H _(Lp1)H _(Lp2))}  (4)

{right arrow over (H _(L2) H _(L3))}=R _(θ2) R _(θ1){right arrow over (H_(Lp1) H _(Lp2))}  (5)

While two-dimensional rotation of the joint is described forsimplification in this example, three-dimensional rotation can be alsotransferred in a similar manner. As described above, when only therotation of the joint is transferred, only a bent state of the joint canbe changed without changing the position of the reference joint.

The skeleton motion of the model is transferred onto the skeleton dataabout the learner in the above-described manner, so that the skeletondata about the learner can perform the motion of the model. In otherwords, the information processing apparatus 100 can generate the motioninformation associated with the skeleton data representing the physicalmakeup of the learner, based on the skeleton data representing thephysical makeup of the model and the motion information representing themotion of the plurality of joints of the model. The informationprocessing apparatus 100 can display, on the display unit, the imagebased on the motion information associated with the skeleton data aboutthe learner and the image representing the motion of the learner basedon imaging of the learner by the camera (image based on data acquired byprocessing in step S601). These images all correspond to the physicalmakeup of the learner, where one of the images represents the motion ofthe learner and the other image represents the motion of the model.

In the case where the image representing the motion of the model and theimage representing the motion of the learner are superimposed anddisplayed, the information processing apparatus 100 performs thepositioning and the time synchronization on the skeleton data tofacilitate comparison between the image representing the motion of themodel and the image representing the motion of the learner. In a casewhere a deficiency occurs on the data as a result of estimation of theshape data and the skeleton data with use of the data based on theimaging, processing to interpolate the deficiency is performed. Theprocessing in step S603 to perform the positioning, the timesynchronization, and the deficiency interpolation on the skeleton datais described with reference to FIG. 8 and FIG. 10. FIG. 8 illustratesdetailed flow of the processing in step S603.

In step S801, the position correction unit 1006 performs positioning onthe skeleton data representing the motion of the model and the skeletondata about the learner, and outputs resultant data to the timesynchronization unit 1007. Rough positioning can be performed using areference joint H near the pelvis of a human including many skeletonbranches, as a reference point of the positioning of the skeleton data.In other words, the positioning for display is performed such that arotation axis and a position of the reference joint HL of the learnerand a rotation axis and a position of the reference joint HE of themodel are superimposed on each other. A specific joint as a reference ofthe positioning may not be the joint near the pelvis. For example, in acase where motion of a wrist of a baseball pitcher is compared,positioning can be performed on a position of a joint of the wrist. Forexample, the positioning can be performed based on a centroid of each ofthe learner and the model without being limited to the joint. In a casewhere the automatic check box is checked when the positioning button 306is pressed, the position correction unit 1006 automatically performs thepositioning based on the reference joint. In a case where the automaticcheck box is not checked but a specific joint is designated, theposition correction unit 1006 performs the positioning based on thedesignated joint.

In step S802, the time synchronization unit 1007 performs the timesynchronization between the motion of the model and the motion of thelearner, and outputs the synchronized operation data to the deficiencyinterpolation unit 1008. For example, in a case where a moving imagerepresenting motion of golf swing by the model and a moving imagerepresenting motion of golf swing by the learner are displayed, the timesynchronization processing is performed such that specific time pointssuch as moments of impact in both images are synchronized with eachother. More specifically, the synchronization processing is performedbased on inflection points in the skeleton motion illustrated in theanalyzed data display area 302. In a case where the skeleton motion ofthe learner based on the captured data includes noise, smoothingprocessing to eliminate the noise is performed before determination ofthe inflection points in the skeleton motion. As a smoothing method in atime direction, for example, a mean value filter, a median filter, and aGaussian filter can be used. The time synchronization unit 1007calculates a plurality of times t that satisfy an expression (6) withrespect to smoothed operation data S(f(t)), thereby determining theinflection points. The processing can be performed on all of the jointsor can be performed on one or more predetermined joints.

$\begin{matrix}{\frac{{dS}\left( {f_{L}(t)} \right)}{dt} = 0} & (6)\end{matrix}$

The time synchronization unit 1007 performs the processing also on theoperation data about the model to calculate inflection points in themotion of the model. The time synchronization unit 1007 sets a starttime and an end time of the motion of the learner to coincide with astart time and an end time of the motion of the model before matchingthe inflection points based on these results. The time synchronizationunit 1007 can eliminate, from the comparison target, unnecessary datasuch as a time series that is included in the operation data about themodel but is not included in the operation data about the learner, and atime series that is not included in the operation data about the modelbut is included in the operation data about the learner. After thisprocessing is appropriately performed, the time synchronization unit1007 associates the inflection points in the motion of the model and theinflection points in the motion of the learner with each other. Examplesof the method of associating the inflection points include a methodusing change of inclination near each of the inflection points in thegraph representing the motion of the joint. The inclination near each ofthe inflection points is calculated from a differential equation, andchange c in the inclination with respect to the time t of each of theinflection points is recorded (expression (7)). A matrix in which thechange in the inclination corresponding to each time t is recorded isdenoted by T. For example, in a case of a matrix TE(m, n) in which theinflection points of the model and the change in the inclination at eachof the inflection points are recorded, m=tm and n=ctm are establishedfor an m-th inflection point from the beginning.

$\begin{matrix}{c = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} \left( {\frac{d^{2}{S\left( {f\left( {t - {\Delta \; t}} \right)} \right)}}{{dt}^{2}} > 0} \right)}\bigcap\left( {\frac{d^{2}{S\left( {f\left( {t + {\Delta \; t}} \right)} \right)}}{{dt}^{2}} < 0} \right)} \\{{- 1}\mspace{14mu}} & {{{if}\mspace{14mu} \left( {\frac{d^{2}{S\left( {f\left( {t - {\Delta \; t}} \right)} \right)}}{{dt}^{2}} < 0} \right)}\bigcap\left( {\frac{d^{2}{S\left( {f\left( {t + {\Delta \; t}} \right)} \right)}}{{dt}^{2}} > 0} \right)}\end{matrix} \right.} & (7)\end{matrix}$

The inflection points are matched using the time of each of theinflection points and the change in the inclination before and aftereach of the inflection points. As an example, a case where inflectionpoints are matched from a start time of the operation data is described.In a case where the inflection points are searched from the start time,an inflection point t_(E1) immediately after start of the operation dataabout the model is searched. It is determined whether there is aninflection point near the inflection point t_(E1) in the operation dataabout the learner. In a case where there is only one correspondinginflection point, the inflection point is regarded as a matching pointt_(L1). In a case where there is a plurality of corresponding inflectionpoints, an inflection point having an inclination before and after theinflection point coincident with the inclination of the inflection pointt_(E1) out of the plurality of corresponding inflection points isregarded as a matching point t_(L1). In a case where there is aplurality of inflection points each having an inclination coincidentwith the inclination of the inflection point t_(E1), an inflection pointcloser to the inflection point t_(E1) is regarded as a matching pointt_(L1). After the matching point t_(L1) is found out,expansion/contraction in the time direction is performed. The data abouta section from a time 0 to a time t_(L1) is multiplied by t_(E1)/t_(L1),and the section data is smoothly connected to data about a subsequentsection (expression (8)).

$\begin{matrix}{{f(t)} = \left\{ \begin{matrix}{f_{L}\left( {\frac{t_{L\; 1}}{t_{E\; 1}}t} \right)} & \left( {0 \leq t \leq t_{L\; 1}} \right) \\{f_{L}\left( {t - \left( {t_{E\; 1} - t_{L\; 1}} \right)} \right)} & \left( {t \geq t_{L\; 1}} \right)\end{matrix} \right.} & (8)\end{matrix}$

In a case where there is no matching point, this processing is notperformed, and next inflection point is processed. Thereafter, theprocessing on each section is repeated until the final inflection pointin the motion of the model is processed, thereby performing the timesynchronization. The inflection points may not be necessarily matchedfrom the start time of the motion. The inflection points can be matchedreversely from the end time of the motion, or the time synchronizationcan be performed based on one or more time points each having motionfeature, such as moment of impact in golf.

In step S803, the deficiency interpolation unit 1008 performs thedeficiency interpolation on the skeleton motion of the learner, andoutputs interpolated operation data to the data comparison unit 1009 andthe data editing unit 1010. The time synchronization of the skeletonmotion is performed based on the inflection points in the skeletonmotion. Therefore, if there is a deficiency in the learner data at atime point between the inflection points, the interpolation is performedusing the model data at the same time point. A start time of thedeficiency is denoted by t_(Lr1), an end time is denoted by t_(Lr2), anddisplacement of the joint of the model and the joint of the learner atthese time points are denoted by θ_(Lr1), θ_(Lr2), θ_(Lr1), and θ_(Er2).The deficiency is interpolated by an expression (9).

$\begin{matrix}{{f_{L}(t)} = {{\frac{{\theta_{{Lr}\; 1} - \theta_{{Lr}\; 2}}}{{\theta_{{Er}\; 1} - \theta_{{Er}\; 2}}}{f_{E}(t)}} - {\left( {\theta_{{Lr}\; 1} - \theta_{{Lr}\; 2}} \right)\left( {t_{Lr} \leq t \leq t_{{Lr}\; 2}} \right)}}} & (9)\end{matrix}$

The interpolation is performed on each deficiency in the above-describedmanner.

The processing relating to data display in step S604 is described withreference to FIG. 9 and FIG. 10. FIG. 9 illustrates detailed flow of theprocessing in step S604. In step S901, the data comparison unit 1009compares the operation data about the model and the operation data aboutthe learner, and outputs a result of the comparison. The outputcomparison result includes numerical value information illustrated inthe edited data display area 502 and image information illustrated inthe image display area 501 in FIG. 5. Only one of the numerical valueinformation and the image information can be output. In the output ofthe image information, data is displayed in a manner enabling thelearner to easily recognize a difference. Examples of the datacomparison method include a method of calculating an outline differencebetween persons as viewed from an optional viewpoint. The learner data(data representing shape and motion of learner) and the model data (datarepresenting shape in case where learner performs motion of model) eachinclude a three-dimensional shape. Therefore, it is possible tocalculate a shape to be imaged by a virtual camera placed at an optionalviewpoint in a three-dimensional space. For example, determination abouta collision between beams emitted from pixels of the virtual camera andthe shape data about the object is performed, and pixels in which thecollision occurs are labeled. As a result, the outline of the object iscalculated. Here, when the outline image of the learner data as viewedfrom a virtual viewpoint H is denoted by M_(L,H), and the outline imageof the model data is denoted by M_(E,H), the data comparison unit 1009calculates the virtual viewpoint H satisfying an expression (10) anddisplays the image viewed from the virtual viewpoint H.

Ĥ=arg max∥M _(E,H) −M _(L,H)∥  (10)

In this example, as the viewpoint determined based on the motioninformation on the learner and the motion information on the model, theimage of the viewpoint at which the motion difference between the modeland the learner is the largest is displayed, but the viewpointcorresponding to the display image is not limited thereto. For example,an image from a viewpoint at which the motion difference is the smallestcan be displayed, or images from a plurality of viewpoints, for example,a side view, a top view, and a front view, can be displayed. In additionto the image for direct comparison between the learner data and themodel data, the image representing comparison between the motion of theedited data and the motion of the learner or between the motion of theedited data and the motion of the model can be displayed after the datais edited in step S902.

In step S902, the data editing unit 1010 edits the operation data outputin step S803, and outputs the edited operation data to the datacomparison unit 1009 and the display method setting unit 1011. Theoperation data can be edited based on a user operation on the imagedisplay area 501 or based on a user operation on the graph in the editeddata display area 502. In a case where the operation is performed on theimage display area 501, the user can edit the operation data about thelearner by three-dimensionally operating the selected joint. In a casewhere the graph in the edited data display area 502 is edited, the usercan edit the operation data about the learner by operating atwo-dimensional angle displacement graph of the joint. In the presentexemplary embodiment, in the case where the operation data is edited,the edited operation data is separately generated while the originaloperation data is held. The data editing unit 1010 can automaticallyedit the operation data based on one or more of the learner data or themodel data.

In step S903, the display method setting unit 1011 sets a method ofdisplaying the data output in step S902. The method of displaying thedata is set based on an operation on each of the display setting area503 for model data, the display setting area 504 for estimated data, andthe display setting area 505 for edited data. For example, eitherdisplaying or not displaying the model data is selected through a modeldata check box in the display setting area 503 for model data. In a casewhere the model data is displayed, one or more of the shape data or theskeleton data is displayed. Transparency can be set for each displaydata, and the display data can be transparently displayed. A displayformat of the display data can be set, in which, for example, the datais displayed with texture or only the outline is displayed.

After the processing in step S604 ends, the imaging determination unit1001 determines in step S605 whether to perform imaging of the movingimages again based on the input to the user input unit 1012. When thedata estimation shifting button 507 is pressed, the screen is returnedto the data estimation GUI 300, and the imaging is performed again inresponse to pressing of the imaging button 303. In a case where theimaging is not to be performed again, the processing ends.

When the learner improves the form using the analysis system 10according to the present exemplary embodiment, the learner can operatethe information processing apparatus 100 after the learner's form iscaptured, and compare the operation data about the model and thelearner's operation data. Alternatively, the comparison can be performedwhile the imaging is performed. In the case where the comparison isperformed while the imaging is performed, the information processingapparatus 100 performs the shape estimation and the skeleton estimationin real time using the data based on the imaging, and displays theimage. In this case, the learner corrects the learner's form whileviewing the display image, thereby gradually reducing a difference fromthe model. For example, the information processing apparatus 100 canfirst display the image from the viewpoint at which the motiondifference between the learner and the model is the greatest, and afterthe difference at the viewpoint is reduced to a certain degree via formcorrection, the information processing apparatus 100 can change thedisplay image to an image at a viewpoint at which the motion differenceis the second greatest. When the motion difference between the learnerand the model is reduced to a certain degree or more at all of theviewpoints, the information processing apparatus 100 can display animage indicating that the motion difference has been reduced on thedisplay unit.

As described above, the information processing apparatus 100 accordingto the present exemplary embodiment acquires first physical makeupinformation representing a physical makeup of a first object (e.g.,learner). The first physical makeup information is information generatedbased on images obtained by imaging the first object by the plurality ofcameras in the plurality of directions. The information processingapparatus 100 acquires second physical makeup information representing aphysical makeup of a second object (e.g., model) and motion informationrepresenting motion of a plurality of parts of the second object. Theinformation processing apparatus 100 generates motion informationassociated with the first physical makeup information based on thesecond physical makeup information and the motion information relatingto the second object. According to the above-described configuration, itis possible to provide a result of applying the motion of the secondobject to the physical makeup of the first object different from thesecond object. For example, an image representing a state where thelearner performs the motion of the model can be displayed on the displayunit.

The information processing apparatus 100 displays, on the display unit,the normal image of the learner and the image of the learner performingthe motion of the model, which enables the user to easily compare themotion of the learner and the motion of the model. The informationprocessing apparatus 100 performs one or more of the positioning or thetime synchronization between the motion data about the learner and themotion data about the model, and superimposes and displays the motionimage of the model and the motion image of the learner, whichfacilitates the comparison between the motion of the learner and themotion of the model. The information processing apparatus 100 displaysthe image from the viewpoint at which the motion difference between thelearner and the model is the greatest, which further facilitates thecomparison between the motion of the learner and the motion of themodel. The learner can easily improve the learner's motion by viewingthe image displayed in such a manner, as compared with the existingtechnology. The information processing apparatus 100 also outputs aresult (e.g., angle information on a joint) of the comparison betweenthe motion of the learner and the motion of the model, which enables thelearner to numerically grasp the difference from the model.

In the present exemplary embodiment, the case where the imagerepresenting the motion of the model and the image representing themotion of the learner are superimposed and displayed has been described,but the display mode is not limited thereto. For example, the imagerepresenting the motion of the model and the image representing themotion of the learner can be arranged and displayed side by side. Amoving image representing the motion of the model and a still image ofthe learner can be displayed. Just the image of the learner performingthe motion of the model that is acquired by applying the motion of themodel to the physical makeup of the learner can be displayed.

In the present exemplary embodiment, the case where the physical makeupinformation representing the physical makeup of the object includes thethree-dimensional shape information on the object and the skeletoninformation representing the positions of the joints has been described,but the contents of the physical makeup information are not limitedthereto. For example, the physical makeup information on the learner andthe model acquired by the information processing apparatus 100 caninclude just the three-dimensional shape information or the skeletoninformation. For example, the information processing apparatus 100 canacquire data representing the two-dimensional shape of the learner asthe physical makeup information, and can apply the motion data about themodel to the data representing the two-dimensional shape of the learner.For example, the physical makeup information can be one or more ofinformation relating to a body height of the object, informationrelating to a weight, and information relating to a figure (such asskinny figure and obese figure), etc. The information processingapparatus 100 can receive input of such physical makeup information onthe learner, correct the operation data about the model based on theinput information, and display the image representing the correctedmotion. When the operation data about the model is applied to theinformation such as the skeleton data about the learner acquired basedon the imaging, an image close to an image of the actual learnerperforming the motion of the model can be acquired. According to themethod of correcting the motion data about the model based on thephysical makeup information input by the user operation without usingthe captured image, it is possible to reduce a processing amount of thesystem and to reduce effort of the user for the imaging.

In the present exemplary embodiment, the case where the motion of thelearner and the motion of the model are compared has been described, butthe above-described configuration can be applied to compare the shape ofthe learner and the shape of the model. Methods similar to thosedescribed above can be used for the positioning, the timesynchronization, the deficiency interpolation, and the data displayusing the GUI.

The above-described exemplary embodiment can provide information usefulfor a person to achieve motion of another person.

Other Embodiments

Embodiment(s) can also be realized by a computer of a system orapparatus that reads out and executes computer executable instructions(e.g., one or more programs) recorded on a storage medium (which mayalso be referred to more fully as a ‘non-transitory computer-readablestorage medium’) to perform the functions of one or more of theabove-described embodiment(s) and/or that includes one or more circuits(e.g., application specific integrated circuit (ASIC)) for performingthe functions of one or more of the above-described embodiment(s), andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s) and/or controlling the one or morecircuits to perform the functions of one or more of the above-describedembodiment(s). The computer may comprise one or more processors (e.g.,central processing unit (CPU), micro processing unit (MPU)) and mayinclude a network of separate computers or separate processors to readout and execute the computer executable instructions. The computerexecutable instructions may be provided to the computer, for example,from a network or the storage medium. The storage medium may include,for example, one or more of a hard disk, a random-access memory (RAM), aread only memory (ROM), a storage of distributed computing systems, anoptical disk (such as a compact disc (CD), digital versatile disc (DVD),or Blu-ray Disc (BD)™), a flash memory device, a memory card, and thelike.

While exemplary embodiments have been described, it is to be understoodthat the disclosure is not limited to the disclosed exemplaryembodiments. The scope of the following claims is to be accorded thebroadest interpretation so as to encompass all such modifications andequivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2018-125011, filed Jun. 29, 2018, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An information processing apparatus, comprising:a first acquisition unit configured to acquire first physical makeupinformation representing a physical makeup of a person as a firstobject, the first physical makeup information generated based on imagesobtained by imaging the first object with a plurality of imagingapparatuses in a plurality of directions; a second acquisition unitconfigured to acquire second physical makeup information representing aphysical makeup of a person as a second object and motion informationrepresenting motion of a plurality of parts of the second object; and ageneration unit configured to generate motion information associatedwith the acquired first physical makeup information based on theacquired second physical makeup information and the acquired motioninformation representing the motion of the plurality of parts of thesecond object.
 2. The information processing apparatus according toclaim 1, further comprising a display control unit configured todisplay, on a display unit, a first image representing motion of thefirst object based on the imaging of the first object and a second imagebased on the generated motion information associated with the acquiredfirst physical makeup information.
 3. The information processingapparatus according to claim 2, wherein the first image is a capturedimage obtained by imaging the first object or an image based on motioninformation on the first object generated based on the images obtainedby imaging the first object with the plurality of imaging apparatuses inthe plurality of directions.
 4. The information processing apparatusaccording to claim 2, wherein the second image is an image representinga three-dimensional shape of an object or a skeleton image of theobject.
 5. The information processing apparatus according to claim 2,wherein the display control unit superimposes and displays the firstimage and the second image.
 6. The information processing apparatusaccording to claim 5, wherein the display control unit displays thefirst image and the second image such that a position of a specificjoint in the second image and a position corresponding to the specificjoint in the first image are superimposed.
 7. The information processingapparatus according to claim 2, wherein the first image and the secondimage are moving images, and wherein the display control unit displaysthe first image and the second image such that a specific time point ofthe first image and a specific time point of the second image aresynchronized, the specific time point of the first image being specifiedby motion information on the first object generated based on the imagesobtained by imaging the first object with the plurality of imagingapparatuses in the plurality of directions, and the specific time pointof the second image being specified by the generated motion informationassociated with the acquired first physical makeup information.
 8. Theinformation processing apparatus according to claim 2, wherein thedisplay control unit displays the first image and the second image at aviewpoint determined based on motion information on the first objectgenerated based on the images obtained by imaging the first object withthe plurality of imaging apparatuses in the plurality of directions andthe generated motion information associated with the acquired firstphysical makeup information.
 9. The information processing apparatusaccording to claim 2, wherein the display control unit displays, on thedisplay unit, an image corresponding to a display mode selected from aplurality of display modes including a first display mode in which thefirst image and the second image are displayed and a second display modein which the first image and a third image representing motion of thesecond object are displayed.
 10. The information processing apparatusaccording to claim 1, further comprising an output unit configured tooutput a result of comparison between motion information on the firstobject generated based on the images obtained by imaging the firstobject with the plurality of imaging apparatuses in the plurality ofdirections and the generated motion information associated with theacquired first physical makeup information.
 11. The informationprocessing apparatus according to claim 1, wherein the acquired firstphysical makeup information and the acquired second physical makeupinformation include at one or more of information representing positionsof joints of an object, information relating to a body height, orinformation relating to a weight.
 12. The information processingapparatus according to claim 1, wherein the generated motion informationis information representing motion of joints of an object.
 13. Theinformation processing apparatus according to claim 1, wherein theacquired second physical makeup information and the acquired motioninformation representing the motion of the plurality of parts of thesecond object are information generated based on images obtained byimaging the second object with a plurality of imaging apparatuses in aplurality of directions.
 14. An information processing apparatus,comprising: a first acquisition unit configured to acquire firstphysical makeup information representing a physical makeup of a personas a first object; a second acquisition unit configured to acquiresecond physical makeup information representing a physical makeup of aperson as a second object and motion information representing motion ofa plurality of parts of the second object, the motion informationgenerated based on images obtained by imaging the second object with aplurality of imaging apparatuses in a plurality of directions; and ageneration unit configured to generate motion information associatedwith the acquired first physical makeup information based on theacquired second physical makeup information and the acquired motioninformation representing the motion of the plurality of parts of thesecond object.
 15. The information processing apparatus according toclaim 14, further comprising a display control unit configured todisplay, on a display unit, a first image representing motion of thefirst object based on imaging of the first object with an imagingapparatus, and a second image based on the generated motion informationassociated with the acquired first physical makeup information.
 16. Theinformation processing apparatus according to claim 14, furthercomprising an output unit configured to output a result of comparisonbetween motion information on the first object generated based on theimages obtained by imaging the first object with the plurality ofimaging apparatuses in the plurality of directions and the generatedmotion information associated with the acquired first physical makeupinformation.
 17. An information processing method, comprising: acquiringfirst physical makeup information representing a physical makeup of aperson as a first object, the first physical makeup informationgenerated based on images obtained by imaging the first object with aplurality of imaging apparatuses in a plurality of directions; acquiringsecond physical makeup information representing a physical makeup of aperson as a second object and motion information representing motion ofa plurality of parts of the second object; and generating motioninformation associated with the acquired first physical makeupinformation based on the acquired second physical makeup information andthe acquired motion information representing the motion of the pluralityof parts of the second object.
 18. The information processing methodaccording to claim 17, further comprising displaying, on a display unit,a first image representing motion of the first object based on imagingof the first object with the plurality of imaging apparatuses, and asecond image based on the generated motion information associated withthe acquired first physical makeup information.
 19. The informationprocessing method according to claim 17, further comprising outputting aresult of comparison between motion information on the first objectgenerated based on the images obtained by imaging the first object withthe plurality of imaging apparatuses in the plurality of directions andthe generated motion information associated with the acquired firstphysical makeup information.
 20. A non-transitory computer-readablestorage medium storing a program causing a computer to execute aninformation processing method, the information processing methodcomprising: acquiring first physical makeup information representing aphysical makeup of a person as a first object, the first physical makeupinformation generated based on images obtained by imaging the firstobject with a plurality of imaging apparatuses in a plurality ofdirections; acquiring second physical makeup information representing aphysical makeup of a person as a second object and motion informationrepresenting motion of a plurality of parts of the second object; andgenerating motion information associated with the acquired firstphysical makeup information based on the acquired second physical makeupinformation and the acquired motion information representing the motionof the plurality of parts of the second object.